Structured Perceptron with Inexact Search
نویسندگان
چکیده
Most existing theory of structured prediction assumes exact inference, which is often intractable in many practical problems. This leads to the routine use of approximate inference such as beam search but there is not much theory behind it. Based on the structured perceptron, we propose a general framework of “violation-fixing” perceptrons for inexact search with a theoretical guarantee for convergence under new separability conditions. This framework subsumes and justifies the popular heuristic “early-update” for perceptron with beam search (Collins and Roark, 2004). We also propose several new update methods within this framework, among which the “max-violation” method dramatically reduces training time (by 3 fold as compared to earlyupdate) on state-of-the-art part-of-speech tagging and incremental parsing systems.
منابع مشابه
Online Learning for Inexact Hypergraph Search
Online learning algorithms like the perceptron are widely used for structured prediction tasks. For sequential search problems, like left-to-right tagging and parsing, beam search has been successfully combined with perceptron variants that accommodate search errors (Collins and Roark, 2004; Huang et al., 2012). However, perceptron training with inexact search is less studied for bottom-up pars...
متن کاملIncremental Joint Extraction of Entity Mentions and Relations
We present an incremental joint framework to simultaneously extract entity mentions and relations using structured perceptron with efficient beam-search. A segment-based decoder based on the idea of semi-Markov chain is adopted to the new framework as opposed to traditional token-based tagging. In addition, by virtue of the inexact search, we developed a number of new and effective global featu...
متن کاملAn inexact alternating direction method with SQP regularization for the structured variational inequalities
In this paper, we propose an inexact alternating direction method with square quadratic proximal (SQP) regularization for the structured variational inequalities. The predictor is obtained via solving SQP system approximately under significantly relaxed accuracy criterion and the new iterate is computed directly by an explicit formula derived from the original SQP method. Under appropriat...
متن کاملOptimal Shift-Reduce Constituent Parsing with Structured Perceptron
We present a constituent shift-reduce parser with a structured perceptron that finds the optimal parse in a practical runtime. The key ideas are new feature templates that facilitate state merging of dynamic programming and A* search. Our system achieves 91.1 F1 on a standard English experiment, a level which cannot be reached by other beam-based systems even with large beam sizes.1
متن کاملEasy-First POS Tagging and Dependency Parsing with Beam Search
In this paper, we combine easy-first dependency parsing and POS tagging algorithms with beam search and structured perceptron. We propose a simple variant of “early-update” to ensure valid update in the training process. The proposed solution can also be applied to combine beam search and structured perceptron with other systems that exhibit spurious ambiguity. On CTB, we achieve 94.01% tagging...
متن کامل